Modeling Dynamic Inconsistency with a Dynamic Reference Point

An experiment was conducted to test a rational principle of decision making called dynamic consistency. Decision makers were presented with a sequence of two gambles, and they were asked to make two decisions. Before the first gamble was played, they made a plan for the second gamble contingent on the outcome of the first gamble; then they played the first gamble and were asked to make a final decision regarding the second gamble after experiencing the first outome. Violations of dynamic consistency were observed, e.g., decision makers planned to the gamble a second time if the first gamble was won; but when in fact the first gamble was won, they switched and chose not to play the gamble a second time. Two models of dynamic inconsistency were compared at the individual level: One assumes that experience shifts the reference point and changes the utility associated with the final choice; another assumes that experience changes the subjective probability associated with the final choice. The reference point model provided the best account for the majority of participants. Modeling Dynamic Inconsistency 3 Most real life decisions require decsion makers to make plans for an uncertain future before they can decide how to act in the present. For example, a student must plan for a future job when deciding courses to take now in college; or a CEO must plan ahead for future markets when deciding products to develop now in the company. The generally prescribed procedure for planning a path of future actions entails backwards induction (c.f. Keeny & Raiffa, 1976; Raiffa, 1968; Von Vinterfeldt & Edwards, 1986), and one of the central assumptions of backward induction is a principle called dynamic consistency (Machina, 1989; Sarin & Wakker, 1998). According to the this principle, mere experience with the events upon which a plan was based should not change the preferences used to form the original plan. Empirical Tests of Dynamic Inconsistency Barkan and Busemeyer (1999) demonstrated the phenomenon of dynamic inconsistency utilizing a sequential gambling paradigm (originally developed by Tversky and Shafir, 1992). Barkan and Busemeyer (1999) presented DMs with two stage decision problems consisting of two sequential gambles (see Figure 1). The first gamble was obligatory and the DMs could either win or lose points with equal probability. Before playing the first gamble, the DMs had to make a plan as to whether or not they would take a second identical gamble. DMs made planned choices for the second gamble contingent on each possible outcome (gain, loss) of the first gamble. After playing the first gamble and actually experiencing its outcome, DMs then made a second final choice regarding the second gamble. Modeling Dynamic Inconsistency 4 Barkan and Busemeyer (1999) found that on 20% of the times DM’s final choices were inconsistent with their planned choices. Moreover, these changes of plans were systematic in their direction contingent on the experienced outcome. One direction of inconsistency showed risk aversion after experiencing gain. That is, when considering winning the first gamble, a DM made a planned choice to take the second gamble. However, after experiencing the anticipated gain, that DM changed his preference and made a final choice not to take the second gamble. Another direction of inconsistency showed risk seeking after experiencing loss. That is, when considering losing the first gamble, a DM made a planned choice not to take the second gamble. However, after experiencing the anticipated loss, that same DM changed his/her preference and made a final choice to take the second gamble. Three Alternative Explanations Barkan and Busemeyer (1999) considered three alternative explanations for these findings. One explanation is that dynamic inconsistency simply reflects choice inconsistency -that is random fluctuations in preferences for the same gamble presented twice. However, Barkan aand Busemeyer (1999) proved that choice inconsistency (alone) cannot explain the systematic directions or pattern of inconsistency that they found. A second explanation, following Tversky and Shafir (1992), is that experience changes the reference point used to evaluate the utility of the second gamble. This explanation assumes an asymmetric utility function (e.g. Prospect Theory, Kahneman & Tversky, 1979). When planning, the DM evaluates the second gamble against a neutral reference point. Experiencing Modeling Dynamic Inconsistency 5 a gain moves the reference point away from the neutral point towards the concave part of the utility function. When making the final choice, the DM reevaluates the second gamble against the new reference point. The reevaluation would cause the same gamble to seem less attractive than before and may lead the DM to reject it. Experiencing a loss moves the reference point in the opposite direction towards the convex part of the utility function. Re-evaluating the second gamble against the new reference point would make it seem more attractive than before and may lead the DM to accept it. A third explanation is based on changes in subjective probability rather than the utility associated with the second gamble. According to this explanation, experience triggers a change in the subjective probability in a way resembling the gambler’s fallacy. When planning, the DM only considers the stated probabilities for winning and losing the second gamble. However, experiencing a gain in the first gamble would lead to a decrease in the subjective probability associated with another gain (in the second gamble). Re-evaluating the second gamble with decreased subjective probability for winning would make the same gamble seem less attractive than before and may lead the DM to reject it. The opposite would happen after experiencing a loss, since the subjective probability for another loss (in the second gamble) would decrease and the subjective probability for a gain would increase. Reevaluation of the second gamble would make it seem more attractive than before and may lead the DM to accept it. Barkan and Busemeyer (1999) argued in favor of the changing reference point model over the changing subjective probability model. However, the design of their initial study did not provide sufficient leverage to Modeling Dynamic Inconsistency 6 rigorously compare these two explanations. Thus, the cause of the systematic directions of inconsistency remains to be determined more convincingly. New Experimental Tests In order to test these two hypotheses more rigorously, we extended the design of the Barkan and Busemeyer (1999) study by examining the sequential gambling experiment with a broader set of 16 decision problems. Each decision problem consisted of two identical gambles. Each gamble gave 50% chance to win or lose points. The Expected Values of the gambles ranged from –10 to 50 in steps of 10. The first gamble in each decision problem was obligatory. One hundred DMs were asked to make planned choices as to whether they would take a second identical gamble contingent on winning and losing the first gamble. After the first gamble was played and its outcome was known, DMs were asked to make a second (final) choice regarding the second gamble. One of their choices (planned or final) was sampled at random to decide whether or not the second gamble would take place. Each Expected Value was represented with two decision problems (i.e. two different gambles). In order to control for the outcome of the first gamble for each Expected Value, the first gamble in one decision problem was won and the first gamble in the other decision problem was lost. The set of 16 gambles was counterbalanced and replicated twice. Points were translated to monetary payoff and subjects were paid according to their earnings in four decision problems sampled at random. Choice Probability Results Table 1 presents the probabilities of planned and final choices to take the second gamble. The upper half of Table Modeling Dynamic Inconsistency 7 1 shows the decision problems in which the first gamble was won. When DMs experienced gain, the probability of final acceptance was always lower than the probability of planned acceptance. The lower half of Table 1 shows the decision problems in which the first gamble was lost. When DMs experienced loss, the probability of final acceptance was always higher than the probability of planned acceptance. Dynamic Inconsistency For each decision problem (i.e. trial) the choices of each DM were recorded as either dynamically consistent or inconsistent. Choices were recorded as dynamically consistent when planned and final choices were identical. One consistent case was when both planned and final choices were to take the second gamble. Another consistent case was when both planned and final choices were to reject the second gamble. Choices were recorded as dynamically inconsistent when the DM’s planned choice differed from his/her final choice. The findings showed that the over all proportion of dynamically inconsistent choices was 0.19. When the first gamble was won, the proportion of risk-aversion inconsistencies (planned acceptance and final rejection) was 0.12. The proportion of the risk-seeking inconsistencies (planned rejection and final acceptance) was only 0.05. When the first gamble was lost, dynamically inconsistent choices indicated an opposite pattern. The proportion of risk-aversion inconsistencies was 0.07, and the proportion of risk-seeking inconsistencies was 0.14 (see Figure 2). These findings replicate and extend Barkan and Busemeyer’s (1999) earlier findings, regarding both the over all proportion of dynamically inconsistent Modeling Dynamic Inconsistency 8 trials and the systematic directions of preference reversals contingent on the experienced outcome. Modeling Dynamic Inconsistency The three explanations for dynamic inconsistency are rigorously tested below by formal model comparisons. First we present a general model for the two stage choices; second, we present a probabilistic model that describes the choice process within each stage, third, we incorporate different assumptions into the subjective probabilities and utilities to represent the three alternative explanations. Then the parameters of all three models are estimated by maximum likelihood methods separately for each participant’s data; and finally, the models are compared using chi square lack of fit statistics. General Model. The general model describes the joint probabilities of the four possible pairs that can be obtained from the planned and final decisions. The symbol PG denotes the choice "plan to choose gamble," PC denotes "plan to choose the certain outome," FG denotes "choose gamble on final choice," and FC denotes "choose the certain outcome on final choice." The joint probability of the planning to choose the gamble but finally choosing the sure thing is symbolized as Pr(PG&FC;), and likewise for the other three joint probabilities. The marginal probability of planning to choose the gamble is symbolized as Pr(PG), and likewise, Pr(FC) denotes the marginal probability for finally taking the sure thing. The general model assumes that the final decision can be made by one of two processes: One is to simply recall and repeat the planned choice, Modeling Dynamic Inconsistency 9 and the other is to forget the plan and make a new independent choice at the final stage. The probability of recalling and repeating the previous choice is represented by a parameter denoted m. On the basis of this general model, the two stage choice probabilities are given by Equations 1a -1d. Pr(PG&FG;) = Pr(PG)⋅ [m + (1-m)Pr(FG)] (1a) Pr(PG&FC;) = Pr(PG)⋅ (1-m)Pr(FC) (1b) Pr(PC&FG;) = Pr(PC)⋅ (1-m)Pr(FG) (1c) Pr(PC&FC;) = Pr(PC)⋅ [m + (1-m)Pr(FC)] (1d) If memory were perfect (m=1), the final choice would be identical to the planned choice and there would be no inconsistent cases. If m=0 (no memory at all) then the final choice is independent of the plan and possiibly inconsistent. When 00, u(C)= r). If the first gamble was lost, the new reference point would be in the loss domain (i.e. if r<0, u(C)=-|r|). Assimilating r to the possible outcomes of the second gamble also changes its expected utility (μ). Modeling Dynamic Inconsistency 11 The two redefined outcomes are g+r and l+r. If either of these redefined outcomes are positive their utilities would be determined using α (i.e. if g+r>0, u(g+r)=(g+r) , if l+r>0, u(l+r)=(l+r) ). If either of the redefined outcomes are negative, their utilities would be determined using β (i.e. if g+r<0, u(g+r)= −|g+r|, if l+r<0, u(l+r)= -|l+r|. In sum, this model has the same number of free parameters (θ, m, α, β) as the Baseline model. However, unlike the Baseline model, this model is forced to predict that P(PG&FC;) ≠ P(PC&FG;), which may be an advantage or a disadvantage, depeding on whether the predicted differences are in the correct direction. Thus the Reference change model may perform better or worse than the Baseline model. Probability-change model. For the planned decision, this model uses the same probabilities and utilities as the Baseline and Reference change models. Thus the probability of choosing the gamble during the planning stage is exactly the same for the all three models. For the final decision, we now allow the subjective probabilities to be affected by the outcome experienced during the first stage. Following a win, the subjective probaiblity of winning again is represented by a free parameter, denoted p. The complementary subjective probability 1-p is associated with winning the second gamble after experiencing a loss in the first gamble. However, according to this model, the utility of the gamble (μ) and the utility of the certain outcome (u(C)=0) are not affected by the experienced outcome. Furthermore, to equate number of free parameters, we use the same exponent for both parts of the utility function (u(g) = g, and u(l) = -| l |). Modeling Dynamic Inconsistency 12 In sum, this model has the same number of free parameters (θ, m, α, and p) as the Baseline and Reference change models. However, unlike the previous two models, this model may or may not predict that P(PG&FC;) = P(PC&FG;), depending on specific parameter values for the subjective probabilities. Thus the reference point model may perform better or worse than either the Baseline model or the Reference change model. Parameter estimation. The four free parameters of each model were estimated separately for each of the 100 participants using the following procedure. The DM’s joint choices for each decision problem (i.e. trial) were recorded with four binary valued variables: Xtt(t)=1 whenever a DM consistently chose the gamble on both stages of a trial; Xnn(t)=1 when a DM consistently rejected the gamble on both stages of a trial; Xtn(t)=1 when a DM planned to choose the gamble but finally rejected the gamble on trial; and Xnt(t)=1 when a DM planned to reject and finally chose the gamble on a trial. On each trial only one of these variables was recorded as 1 (i.e. the observed pattern) and the other three were recorded as zeros. Each model’s predictions followed Equations 1-2 (with the specific parameters described above), and provided four probabilities for each trial. The vector of predicted probabilities P(t) = [ Ptt(t), Pnn(t), Ptn(t), Pnt(t) ] corresponded to the vector of the four observed variables X(t) = [Xtt(t), Xnn(t), Xtn(t), Xnt(t) ]. The parameters were selected to maximize log likelihood for each DM (i.e. minimizing the chi-square lack of fit index). χ = -2⋅Σ t = 1,33 { Xtt(t)⋅ln[Ptt(t)]+Xnn(t)⋅ln[Pnn(t)]+Xtn(t)⋅ln[Ptn(t)]+Xnt(t)⋅ln[Pnt(t)]}. Model comparisons. The predictions provided by the three alternative models are shown in Figure 3. All three models capture the Modeling Dynamic Inconsistency 13 general rate of inconsistency (i.e. Ptn+Pnt). However, for dynamic inconsistency it is crucial to explain the direction of inconsistency (i.e, Ptn-Pnt) dependent on the experienced outcome. As noted earlier, the Baseline model cannnot capture this pattern because it predicts equal rates for the two types of inconsistencies (i.e. Ptn=Pnt) regardless of the experienced outcome. Both Reference-change and Probability-change models capture the directions (and magnitudes) of dynamic inconsistency (i.e. Ptn>Pnt after experienced gain, and Ptn<Pnt after experienced loss). Quantitative comparisons between the Baseline, Reference-change and Probability-change DFT models were based on chi–square differences between each pair of competing models. As can be seen in the table, the _____________________________________ Insert Table 2 about here _____________________________________ Reference-change model fit the individuals better than the Baseline model for a majority of individuals (67%), and the mean difference was significant. The Reference change model was also superior to the Probability-change model for a majority of individuals (68%), and the mean difference was again significant. The Probability change model was no better than the Baseline model (49% favored the former), and the mean difference was not significant. The failure of the Probability-change model is interesting and points to the importance of individual difference analyses. While it captured the overall pattern (averaged across subjects), it failed to account for many of the individual patterns. Modeling Dynamic Inconsistency 14 In sum, the Reference change model provided the best explanation of the pattens of choices produced by the individuals. This model also provided a good fit the the final choice probabilities for each gamble. The choice probability predictions for the Reference change model, averaged across subjects, are shown in the last column of Table 1. The correlation between the observed and predicted proportions is r = .90, which is much higher than that predicted by using only the expected values, which yields r = .72. Individual Differences. Table 3 indicates the means, standard deviations (and possible ranges) for the parameters of the Reference-change model. As can be seen, the mean of the utility parameters α and β are in line with the value function suggested by Prospect Theory (Kahneman & Tversky, 1979). The utility function is concave for gains, convex for losses and the function is steeper for losses than for gains. The mean of the memory parameter m indicates that DM’s recalled their planned choices on approximately half of the trials. Finally, the standard deviations imply considerable individual differences in these parameters. Individual differences in model parameters were correlated with the general inconsistency rates (i.e. Σt Xtn(t)+Xnt(t) for each DM). Two of the model parameters, m and θ, were found to be significantly correlated with the general inconsistency rates of the DMs. The negative correlation between m and general inconsistency (r = -.88, p<.001) indicated that as memory for the planned choice decreases, the amount of general inconsistency increases. The negative correlation between θ and general inconsistency (r = -.39 Modeling Dynamic Inconsistency 15 p<.001) indicated that as the threshold decreases, choice becomes more random, and general inconsistency increases. An individual difference analysis of dynamic inconsistency (i.e. Xtn Xnt) was also performed. The 100 DMs were categorized according to their patterned choice behavior (consistent, nondirectional inconsistency, and directional inconsistency). Table 4 shows the frequencies of the three patterns according to three levels of the memory parameter (χ[4]=29.34 p<.001). As can be seen, consistent behavior was observed for only 13 DMs at the highest level of memory. As memory decreased below m=.65, the number of consistent DMs dropped to zero. Twenty DMs showed a nondirectional type of inconsistency. This pattern dropped markedly as memory decreased bellow m= .35. Sixty-three DMs showed a directional type of inconsistency. The relative frequency of this pattern increased as memory decreased. According to the Reference-change model, the different patterns of choice behavior depend on the individual utility functions (i.e. parameters α and β). Figure 4 gives as example the median utility functions of nondirectional and directional inconsistency groups. Nondirectional inconsistency types are characterized with almost linear utility functions. Directional inconsistency types are characterized with Prospect-Theory-like utility functions. Discussion The model comparisons strongly support the Reference change model over the Probability change and Baseline models as an explanation for the dynamic inconsistency results observed in this experiment. According to this explanation, the reference point dynamically changes as the DM progresses Modeling Dynamic Inconsistency 16 through the planned path in a decision-tree. The change of reference point results in a change of the utility associated with planned and final choices. A gamble may seem attractive while planning in face of a neutral referencepoint. However, after experiencing a gain, the same gamble may become less attractive since the reference point shifts to the concave gain domain of the utility function. Experienced loss would make the same gamble more attractive than before, since the reference point shifts to the convex loss domain of the utility function. The Reference-change model was found to provide a better fit the the individual DM choices than the other two alternative models considered here. The failure of the Baseline model indicated that while memory of planned choice serves as prerequisite for inconsistent choice behavior, it could not explain the systematic directions of inconsistencies. The Probability-change model captured the mean pattern of dynamic inconsistency, but was unable to capture the individual choice patterns any better than the Baseline model. As mentioned earlier, the failure of the Probability-change model is not obvious. Note that both the Reference and Probability-change models allow experience to change the Expected Utility of the gamble (i.e. μ). In the Reference-change model the change in μ is derived from the change in the utilities of possible gain and loss (i.e. u(g), u(l)). In the Probability-change model the change in μ is derived from the change in the subjective probability for winning the second gamble (i.e. p). This analysis of dynamic inconsistency indicates that DMs cannot fully predict how experience will affect their preferences. Planning is made as if the reference point was static, and DMs do not foresee the change in the Modeling Dynamic Inconsistency 17 reference point and the effect it would have on their preferences. The anticipated gains and losses remain isolated and their effect on preferences is only fully assimilated after they have experienced the outcome. A similar phenomenon is known as the endowment effect (Loewenstein & Adler, 1995). This phenomenon shows that DMs value an object much less before owning it as compared to after actually owning it. Loewenstein and Adler suggested the different values associated with that object were based on different reference points of ‘not having’ vs. ‘having’ the object. The present work adds to a growing body of evidence (Busemeyer, Weg, Barkan, Li, Ma, 2000) that suggests that dynamic inconsistency is a pervasive and robust phenomena. These findings challenge the unquestioned use of backward induction by decision analysts for representing an individual's plans in complex multi-stage decision tasks. Modeling Dynamic Inconsistency 18